fix(aws-startup-advisor): refresh GPT-5.4/GPT-5.5 Bedrock endpoint metadata from the public model cards - #583

Problem

The aws-startup-advisor plugin describes GPT-5.4 and GPT-5.5 on Bedrock as reachable only through the OpenAI-compatible (bedrock-mantle) endpoint: no cross-region inference (CRIS) ids, no inference profiles, no bedrock-runtime surface, Chat Completions "unconfirmed", a 272K context window, and no long-context price tier. The canonical catalog skills/shared/ai/models/openai-bedrock.json cited an "internal reference" as the provenance for those rows.

The public Bedrock model cards say otherwise. Both models are served by bedrock-runtime (Responses, Chat Completions, Converse) through us.openai.gpt-5.4 / global.openai.gpt-5.4 and us.openai.gpt-5.5 / global.openai.gpt-5.5, and by bedrock-mantle (Responses, Chat Completions) in-region. Context is 1,050,000 tokens shared with 128,000 max output, and each model has a long-context price tier. Because the same stale statements are repeated in roughly 20 places (shared docs, three vendored copies, phase files in gcp-to-aws / azure-to-aws / bedrock-migration-assessment, llm-to-bedrock helpers, agent prompts, and the tests that lock them in), a startup following the plugin today is told its GPT-5.4/5.5 workload cannot use Converse, Guardrails, or CRIS, and is quoted the wrong context limit and price table.

Fix

Re-source the GPT-5.4 and GPT-5.5 rows in the canonical catalog from the two public model cards and bring every piece of prose, every vendored copy, and every test assertion into line with what the cards state. Concretely:

• Catalog (skills/shared/ai/models/openai-bedrock.json): mantle_openai_chat marked available; a runtime_converse path with CRIS ids; inference_profiles (us, global); region_availability (5.4: us-east-1, us-east-2, us-west-2, us-gov-west-1; 5.5: us-east-1, us-east-2); short and long-context pricing tiers plus the GovCloud rate for 5.4; 1,050,000 / 128,000 context and output limits; provenance is the public card URL. Path-level constraints corrected (Guardrails and application inference profiles on Converse only; implicit prompt caching and structured outputs on Responses only; no server-side tool use; Priority/Flex unsupported).
• Shared docs (ai-openai-to-bedrock.md, ai-migration-guardrails.md, schema-design-aws-ai.md, bedrock-pricing-cache.md): Tier 0 table, region gate, caching notes, Bedrock-native section, long-context table, GovCloud rate. The schema rule now bans bare openai.gpt-* ids paired with converse / runtime, not the models themselves. The pricing prose keeps the existing ~10% cross-region uplift wording for these two because the cards publish only In-Region Standard rows.
• gcp-to-aws, azure-to-aws, bedrock-migration-assessment phase and shared files, and the three vendored copies (re-synced with tools/sync-vendored.py).
• agent-advisor: model-recommend.md documents the engine's current treatment of GPT-5.x as an engine limitation rather than a model fact; openai_model_recommendation.py changes only two message strings (dropped the "GPT-5.x on Mantle rejects Chat Completions" claim and the "Mantle-only GPT-5.x family" wording). No candidate ordering or dispatch logic is touched.
• llm-to-bedrock: resolve-bedrock-model-id.md (Case A/B), openai-to-bedrock.md behavior delta, SKILL.md, and docstrings/comments plus the preflight 404 remedy string in preflight_bedrock.py, iam_policy.py, bedrock_pricing.py. The is_mantle_model / is_mantle_gpt id-form logic is unchanged.
• Agents: llm2bedrock-code-rewriter.md (Converse rule is id-form dependent), llm2bedrock-report-generator.md, llm2bedrock-prompt-evaluator.md.
• Tests: existing GPT-6 review suites and the agent-advisor recommendation tests now assert the corrected facts. New regression tests prove us./global.openai.gpt-5.4|5.5 classify as runtime and bare ids as mantle in all three llm-to-bedrock helpers.

Recording the corrected catalog facts has one visible side effect on recommendations, because the unchanged agent-advisor Chat fallback walks catalog key order and GPT-5.5 / GPT-5.4 precede GPT-6 Astra: a GPT-6 Sol / 6.1 Sol Chat Completions source outside us-east-1 now lands on GPT-5.5 (us-east-2) or GPT-5.4 (us-west-2), and a GPT-5.6 Sol Responses source in us-west-2 lands on GPT-5.6 Terra. Five new regression tests pin those cells so the shift is deliberate and visible, and model-recommend.md calls it out as an engine ordering limitation. It is not fixed here (see below).

What this PR does NOT do

• It does not change endpoint-selection policy. The agent-advisor candidate walk, _SOURCE_MODEL_KEYS, and the llm-to-bedrock id-form dispatch are byte-for-byte the same logic as main. Reordering the Chat fallback so a newer-generation source does not fall back to GPT-5.4/5.5 ahead of GPT-6 Astra, and recognizing GPT-5.4/5.5 as same-model runtime identities, is PR 2 of this stack.
• It does not add new Runtime lanes. No new code paths, selectors, or generated artifacts for the bedrock-runtime GPT-5.4/5.5 route are introduced. That is PR 3 of this stack.
• It does not touch plugin registration, manifests, marketplaces, README.md, or mise.toml.

Scope

Skills: agent-advisor, azure-to-aws, bedrock-migration-assessment, gcp-to-aws, llm-to-bedrock, shared. No new skills.

Files: 46 modified (0 added, 0 deleted), +857 / −390, all under plugins/aws-startup-advisor/.

• skills/shared/ai/: 5 canonical files (models/openai-bedrock.json, ai-openai-to-bedrock.md, ai-migration-guardrails.md, schema-design-aws-ai.md, bedrock-pricing-cache.md).
• Vendored copies: 10 files under agent-advisor/references/vendored/ai/ (2), azure-to-aws/references/vendored/ai/ (4), gcp-to-aws/references/vendored/ai/ (4), byte-identical to the canonical files.
• agent-advisor: model-recommend.md, scripts/openai_model_recommendation.py, scripts/test_openai_model_recommendation.py.
• azure-to-aws: clarify-ai-only.md, design-ai.md, design.md, generate-artifacts-ai.md, generate.md, shared/openai-on-bedrock.md, shared/schema-design-aws-ai.md.
• gcp-to-aws: clarify-ai-only.md, design-ai.md, discover-preview.md, generate-ai.md, shared/openai-on-bedrock.md.
• bedrock-migration-assessment: clarify-ai-only.md.
• llm-to-bedrock: SKILL.md, resolve-bedrock-model-id.md, behavior-delta-detection/references/openai-to-bedrock.md, scripts/preflight_bedrock.py, scripts/iam_policy.py, scripts/bedrock_pricing.py, and their three test files.
• agents/: llm2bedrock-code-rewriter.md, llm2bedrock-prompt-evaluator.md, llm2bedrock-report-generator.md.
• tests/: test_gpt6_review_round2.py, test_gpt6_review_round5.py, test_gpt6_round4_findings.py.

Base: main at 0a218f8c. Two commits: the metadata refresh, and a follow-up that pins and discloses the fallback shift described above.

Not touched: endpoint-selection and dispatch logic (_mantle_candidate_order, _catalog_model_for_path, _SOURCE_MODEL_KEYS, is_mantle_model, is_mantle_gpt), plugin.json, plugin manifests, marketplaces, plugin README.md, mise.toml, heroku-to-aws and the other non-AI skills, .mcp.json.

Validation

Figures in this section are from earlier heads and are kept as history. See "Update after merging main" below for the current head.

Sources: the public Bedrock model cards, read 2026-11-06.

• https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-54.html
• https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-55.html

Local gates, all run from this branch against main at 0a218f8c, all exit 0 unless noted:

• Plugin CI mise -C plugins/aws-startup-advisor run ci: PASS. test 1,395 passed / 222 subtests passed; test:telemetry 311 passed; test:heroku-source-review 44 pass / 0 fail; lint:md 0 errors; lint:model-id OK (19 retired ids enforced); lint:model-staleness fresh (4 days old, 45-day window); lint:model-catalog-drift OK (3 pre-existing warnings, sonnet-5-5, haiku-5-5, gpt-6.1-terra not in catalog, unrelated); lint:startup-model-pricing OK; lint:artifacts 99 checked / 0 errors; lint:vendored-parity 62 files byte-identical; lint:skill-refs 1,482 checked / 0 missing; lint:consent-disclosure OK (6 checked).
• agent-advisor pytest (skills/agent-advisor/scripts): 569 passed / 0 failed (562 at base, 7 new).
• llm-to-bedrock pytest (skills/llm-to-bedrock/scripts): 418 passed / 0 failed (415 at base, 3 new; needs --with httpx --with aws-bedrock-token-generator).
• Root: python3 tools/validate.py "All validations passed"; mise run lint:md 0 issues / 2,763 files; lint:manifests, lint:spec, lint:skill-parity pass; test:tools OK.
• mise run build: the security step fails only on a pre-existing gitleaks finding in upstream commit f8d09fd1 (tests/test_bedrock_migration_assessment_discover.py:118, a claude-3.5-sonnet model string), unrelated to this branch. gitleaks protect --staged over this change set: no leaks found.
• Grep over plugins/ for "mantle-only", "no CRIS", "no inference profile", "no runtime path", "unconfirmed", "internal reference", "no 1M tier" near GPT-5.4 / GPT-5.5 / GPT-5.x: no remaining claims. The only hits are negative test assertions, the GovCloud Opus 5.5 pricing row, and the bare-id rule ("bare proprietary openai.gpt-* ids use Mantle only"), which is still correct.

Review round 2 (head 8c37ef0f, replies to the three inline threads by leon1418 on 6b873f8c):

1. IAM default-project grant (iam_policy.py:29): is_gpt54_55_cris_model() / requires_default_project_grant() added; generate_policy() emits arn:aws:bedrock:{region}:{account-id}:project/default for the us./global.openai.gpt-5.4|5.5 ids and profile ARNs; GPT-6-only wording updated in gcp generate-ai.md, gcp/azure generate-artifacts-ai.md and the report generator. Tests: test_iam_policy.py::TestGpt54Gpt55CrisProjectArn.
2. Application inference profiles are Converse-only (gcp design-ai.md:425): GCP/Azure Design require required_client_api: "converse" when application profiles drive the move on GPT-5.5/5.4, keep the responses default for a region-only move, and record a conflicting Responses-only need as an explicit decision; checklist items added to the GCP checklist and Azure shared schema.
3. Unpublished Geo/Global prices (ai-openai-to-bedrock.md:46): the ~10% premium is scoped to the commercial In-Region option across the canonical guide, both openai-on-bedrock.md fact bases, the report generator and the pricing-cache rows; CRIS plans source a Geo/Global price separately or report the comparison unavailable. bedrock_pricing.lookup() available: false for the CRIS ids pinned by test_bedrock_pricing.py::test_gpt54_55_cris_ids_are_unpriced_not_inferred_from_in_region. This supersedes the third Limitations bullet below.

Text contracts for all three in new tests/test_gpt54_55_runtime_review_round3.py; tools/sync-vendored.py re-run (5 copies). Updated counts: plugin CI test 1,407 passed / 222 subtests (was 1,395); test:telemetry 311; heroku-source-review 44/44; lint:vendored-parity 62 files; lint:skill-refs 1,482. agent-advisor pytest 570 passed; llm-to-bedrock pytest 453 passed (via uv run --project scripts). python3 tools/validate.py passed; tools/sync-vendored.py --check PASS. mise run security still reports only the pre-existing upstream gitleaks false positive at f8d09fd1.

Limitations

• The agent-advisor Chat fallback now routes a GPT-6 Sol / 6.1 Sol Chat Completions source outside us-east-1 to GPT-5.5 or GPT-5.4 (an older generation) because the catalog walk is in key order. This is pinned by tests and documented in model-recommend.md; the ordering fix is deferred to PR 2 of this stack so that this PR stays a metadata correction.
• GPT-5.4 and GPT-5.5 are not yet in _SOURCE_MODEL_KEYS, so a GPT-5.4/5.5 source workload still maps to Claude tiers rather than being recognized as a same-model runtime identity. Also deferred to PR 2.
• The model cards publish only In-Region Standard pricing for these two models (no Geo or Global rows). The prose therefore keeps the existing ~10% cross-region uplift estimate and does not claim price parity.
• lint:model-catalog-drift carries 3 pre-existing warnings and mise run build's security step a pre-existing gitleaks finding, both inherited from main and not addressed here.

Update after merging main

Head e17c00ac. Main was merged into this branch, including #567, #581, #600, #607, #619 and #620. The only conflict was the generated OpenAI model catalog. Main no longer edits skills/shared/ai/models/openai-bedrock.json by hand: it is rendered from skills/shared/ai/models/bedrock-model-registry.json by tools/build-model-catalogs.py, and the vendored copies come from tools/sync-vendored.py. This PR's GPT-5.4/GPT-5.5 corrections were therefore ported into the registry and the catalogs re-rendered; wherever this description says "canonical catalog", the registry is the source and the catalog is its rendered output. The full plugin CI (mise run ci) passed at this head (1,731 tests). The base 0a218f8c and the counts above are from the first revision.

Type of Change

• New skill
• Documentation update
• New plugin
• Bug fix
• CI/CD change
• Other

Checklist

• I have read the CONTRIBUTING guide
• My changes pass python3 tools/validate.py
• I have updated relevant documentation

Startup Advisor review checklist

• I have reviewed the full diff and understand all changes I am submitting.
• I have tested my changes and confirmed they work as intended.
• I have run mise -C plugins/aws-startup-advisor run ci and all checks pass.
• I have disclosed any known issues, limitations, or untested scenarios.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.